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Description 

Field Of The Invention 

[0001] This invention relates generally to handwriting s 
recognition, and more particularly to the selection input 
and editing of discrete continuous segments of hand- 
written input. 

Background of the Invention 10 

[0002] Machine recognition of human handwriting is 
a very difficult problem, and with the recent explosion of 
pen-based computing devices, has become an impor- 
tant problem to be addressed. Machine recognition of is 
human handwriting has various present applications. 
[0003] One example of the current application for ma- 
chine recognition of human handwriting is found in per- 
sonal digital assistants, such as the Newton product and 
other types of pen based computing devices. Typically 20 
these type of products have a touch sensitive screen 
upon which a user can impose handwriting. These de- 
vices then function to digitize the handwritten input, such 
as alphanumeric input, and thereafter process the input 
in an attempt to recognize the information content of the 25 
handwriting. 

[0004] Pursuant to one prior art handwriting recogni- 
tion technique, one makes a best determination as to 
the identity of each alphanumeric character in se- 
quence, with the resulting string of characters compris- 30 
ing the result of the recognition activity. There are a va- 
riety of drawbacks to this approach. It is hindered by the 
difficulty of identifying spatial boundaries of the candi- 
date inputs (in this case alphanumeric characters to be 
recognized. When these boundaries are not located cor- 35 
rectly, it is impossible to recognize the character accu- 
rately, since it will either be lacking pieces or will incor- 
porate extraneous material from adjacent characters. 
[0005] One significant problem with machine recogni- 
tion of human handwriting is the ability to recognize the 40 
end of one input and the beginning of the next input. For 
example, a significant problem exists in locating the end 
of one handwritten input segment, word, or alphanumer- 
ic input, from the beginning of the subsequent handwrit- 
ten input segment, word, or alphanumeric input. Poor 45 
recognition of the handwritten input results in poor, in- 
accurate interpretation of the information content of the 
handwritten input. This problem is accented by poor in- 
put practices of the user or input device, such as poor 
penmanship or handwriting habits. Machine recognition so 
of handwritten input may incorrectly join one or more 
segments of handwritten input into one segment, and 
recognize them as them as a singular discrete segment. 
Alternatively, a user may feel the need to split one or 
more handwritten input segments after such segments ss 
have been entered as a singular discrete handwritten 
input segment. 

[0006] Accordingly, a need exists for a handwriting 
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recognition technique that allows a user or input device 
to enter a selection input edit instruction that permits the 
user, or input device, to split one or more continuous 
segments to form one or more discrete continuous seg- 
ments for recognition and display and thereby provide 
a more accurate interpretation of the information content 
of the handwritten input. This technique is provided by 
the invention as specified in claim 1 . 

Brief Description Of The Drawings 

[0007] 

FIG. 1 Illustrates a flow diagram of operation in ac- 
cordance with a preferred embodiment of the 
present invention. 

FIG. 2 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

FIG. 3 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

FIG. 4 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

FIG. 5 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

FIG. 6 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

FIG. 7 Illustrates a graphical view of an illustrative 
display in accordance with a preferred embodiment 
of the present invention. 

Detailed Description Of The Preferred Embodiments ' 

[0008] Generally, the present invention as disclosed 
provides a user with the option of editing handwritten 
input such that the user may elect to split one or more 
continuous segments, or blocks of ink, to form one or 
more discrete continuous segments. The split discrete 
segments are then presented as individual discrete con- 
tinuous segments and analyzed as individual discrete 
segments by the method of machine recognition of 
handwritten input being employed by a device that func- 
tions to receive handwritten input, such as a Personal 
Digital Assistant (PDA). The recognition of the individual 
discrete segments that result is then displayed to the 
user. 

[0009] In a preferred embodiment of the present in- 
vention the handwritten input is alphanumeric, the writ- 
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ing axis is horizontal, and the writing direction is left to 
right. In accordance with this preferred embodiment, the 
handwritten input and the resultant recognition are dis- 
played concurrently and in close juxtaposition to each 
other. This close juxtaposition allows the userto refer to 
their original handwritten input when correcting errors in 
the processing and recognition of the handwritten input. 
[0010] Typically, handwritten character input is col- 
lected from the user in the form of discrete continuous 
segments. A discrete continuos segment consists of 
one or more pen strokes, where a pen stroke is the mark 
left by a pen during its period of contact with an input 
device such as a digitizing tablet or paper. A stroke is 
represented as a sequence of points sampled at approx- 
imately regular intervals by the input device. Each point 
is described at minimum by an X coordinate and a Y 
coordinate. Strokes may be captured electronically us- 
ing a digitizing tablet, or in alternative embodiments may 
be derived from a scanned or faxed image through a 
process of line detection in the image; such methods of 
capturing input electronically are understood in the art. 
[0011] In the present invention one or more discrete 
continuous segments are the units of handwritten input 
being recognized. Handwritten input is input which is 
captured electronically that includes but is not limited to 
the following: handwritten input; electronic input; input 
captured through pressure, such as stamped input; in- 
put that is received electronically, such as via facsimile, 
pager, or other device. For purposes of explanation of 
the present invention, handwritten input is typically pre- 
sented along a writing axis in a direction which is defined 
as the writing direction. The writing axis is the line along 
which the handwritten input is added. The writing direc- 
tion is the direction in which each subsequent handwrit- 
ten input is added. For example, in English, handwritten 
input is added typically along a horizontal writing axis 
with each subsequent alphanumeric inputfollowing hor- 
izontally after the previous input in a writing direction 
that is left to right. Various other writing axis and writing 
direction alternatives are possible with the implementa- 
tion of the teachings of the present invention. 
[001 2] In a preferred embodiment, the handwriting ax- 
is is horizontal and the handwritten input forms a series 
of words. In this preferred embodiment, the continuous 
discrete segments are entered horizontally from left to 
right. In an alternative preferred embodiment, the hand- 
writing axis is horizontal andthe handwritten inputforms 
a series of separate characters, which may be alphanu- 
meric characters, ideographic characters as found in 
languages such as Chinese, or other forms of charac- 
ters or symbols of written communications. In this alter- 
native embodiment, the output tells whether the discrete 
continuous segments belong to separate characters. In 
another preferred embodiment, the handwriting axis is 
vertical and the handwritten input forms a series of sep- 
arate characters, which may be alphanumeric charac- 
ters, ideographic characters, or other handwritten input. 
In this preferred embodiment, the writing axis is vertical 



and the output tells whether the discrete continuous 
segments belong to separate characters. In another 
preferred embodiment, the handwriting axis is vertical 
and the handwritten input forms a series of separate 

s words, alphanumeric input, or other handwritten input, 
such as a vertical list of words, or numbers. In this pre- 
ferred embodiment, the writing axis is vertical and the 
output tells whether the discrete continuous segments 
belong to separate handwritten input, such as separate 

10 words. 

[0013] As disclosed above and as will be discussed 
further, the present invention demonstrates through the 
disclosure of several of the preferred embodiments that 
the writing axis may exist at any angle and the handwrit- 

15 ten input may be interpreted more generally as corre- 
sponding to discrete elements (including but not limited 
to characters and words) containing one or more dis- 
crete continuous segments. The application of the meth- 
ods described herein to any of various preferred embod- 

20 iments requires only a change in the coordinate system 
used and such modifications can be made in accord- 
ance with the teachings presented. 
[0014] Referring now to the Figures, a personal digital 
assistant (PDA) is generally depicted by reference nu- 

25 meral 1 00. The PDA 1 00 depicted constitutes a generic 
representation and may be comprised for example of a 
Newton® or other pen based computing device. Such 
devices typically include a housing and a touch screen 
upon which input, such as words, can be handwritten 

30 using an appropriate hand manipulated stylus, or other 
input device. Such PDA and pen based computing de- 
vices typically include one or more microprocessors or 
other digital processing devices. These devices com- 
prise computational platforms that can be readily pro- 

35 grammed in accordance with the teachings presented 
herein. It should be understood that, while such person- 
al digital assistants and pen based computing devices 
comprise a ready platform to accommodate the practice 
of applicant's teachings, the teachings presented herein 

40 may be practiced in a variety of other operating environ- 
ments as well. Some examples of such environments 
include computers with digitizing screens, or which are 
connected to a digitizing input surface, or capable of re- 
ceiving faxed or scanned image input, or digital or inter- 
ns active televisions, modems, pagers, or other systems 
with the ability to capture handwritten input and process 
it. 

[0015] Referring now to FIG 1 , a preferred method of 
the present invention is illustrated. The present inven- 

so tion is applicable to one or more handwritten inputs of 
continuous segments. The preferred embodiments of 
the present invention are applicable to handwritten in- 
puts having two or more strokes in a continuous seg- 
ment. The use of three continuous segments S 1 , S 2 , and 

55 s 3 of alphanumeric input are for illustrative purposes on- 
ly. For illustrative purposes S-,, S 2 , and S 3 represent S n , 
S n+1 , and S n+2 respectively. In the preferred method il- 
lustrated in FIGs 1 through 7, handwritten input consist- 
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ing of one or more continuous segments is received by 
an input device 110, such as a personal digital assistant 
100, or other device capable of capturing handwritten 
input. The handwritten input is analyzed by the hand- 
writing recognition method executing on the input 
device , to provide recognition 115 , in this embodiment 
alphanumeric, that represents the corresponding hand- 
written input. In accordance with this preferred embod- 
iment, the recognition of the continuous segments 

5 2 , and S 3 is displayed to the user 120. Preferably, the 
recognition of S 2 , and S 3 is displayed in close juxta- 
position to a digitized representation of the original 
handwritten input of S 1 , S 2 , and S 3 . Once the recognition 
is displayed 120, the user may select to input, or edit, 
125 the displayed recognition 120 of S 2 , and S 3 . If 
the user, or input device, does not enter a selection input 
a recognition is displayed 130 for the one or more con- 
tinuous segments S 2 , and S 3 of handwritten input. 
If, however, the selection input is a split command, or 
instruction, then the continuous segments S 2 , and 
S 3 are analyzed according to the strokes in each seg- 
ment and are split into one or more discrete segments 
dependent if the stroke analysis indicates a gap exists 
among or between the continuous segments S 1 , S 2 , and 

5 3 . Preferably, upon the receipt of a split instruction the 
contiguous segments S 1 , S 2 , and S 3 are parsed accord- 
ing to the strokes T in each segment S., , S 2 , and S 3 (see 
Figs. 2-7). Preferably the strokes are enumerated in 
temporal order, i.e. the order in which the user enters 
them; or input device receives them. This order is not 
necessarily the order that they appear along the writing 
axis. For example if the writing axis is left to right, a 
stroke T n which is written first may appear after a stroke 
T n+1 . For example the upward stroke of the cursive letter 
"i" may be stroke Tn, but the stroke of the "dot" of the 
"I" .which isT n+1 may appear on a the writing axis before 
the stroke T n . 

[0016] In the preferred embodiment of the present in- 
vention, once the continuous segments S-,, S 2 , and S 3 
are parsed into their respective strokes, a map of the 
smallest gap that exist between each stroke and all oth- 
er strokes of the continuous segments Sf,S 2 , and S 3 is 
generated 135 and recorded. For each stroke, a value 
is recorded on the map. The value that is recorded is 
the largest of the values found of smallest gaps for that 
stroke as compared to every other stroke of the seg- 
ments S,, S 2 , and S 3 . As illustrated in FIG 1, 140, the 
value Z is recorded because it is the largest value found 
of those values corresponding to the smallest gaps that 
exist between a particularstrokeT n and all other strokes 
T n+/-1 ...> °f tne continuous segments S-,, S 2 , and S 3 . 
Graphic illustrations of the value of Z are shown in FIGs 
2 - 7. If the value of Z for each stroke is less than zero 
the contiguous segments S v S 2 , and S 3 are concluded 
to be a single continuous handwritten input and is 
diplayedto the user, or input device, as such 130. How- 
ever, if the value of Z, for any stroke T, is greater than 
zero, the one or more continuous segments S 1 , S 2 , and 
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S 3 , are split at the stroke number Tx having the largest 
value Z 150; where T x is defined as the split stroke 
number. If a split stroke number is found, at least one 
discrete segment S x is generated 155, where S x in- 

5 eludes the discrete continuous segments from stroke T 
= Zero to the split stroke number T x . At least a second 
discrete segment S y is generated 160, where S y in- 
cludes the discrete continuous segments from split 
stroke number plus one, T x+1 , to the last stroke T end of 

10 the continuous segments S,, S 2 , and S 3 . In accordance 
with the preferred embodiment of the present invention, 
the discrete continuous segments S x and S y are each 
analyzed and recognized as separate, distinct handwrit- 
ten inputs by the handwriting recognition method exe- 

is cuting on the input device 1 65. The teaching of the 
present invention may be employed by a variety of hand- 
writing recognition methods. The recognition for both S x 
and S y is displayed to the user or input device 1 70. Pref- 
erably, the recognition of S x and S y is displayed in close 

so juxtaposition to a digitized representation of the original 
handwritten input of continuous segments S-,, S 2 , and 
S 3 , or a to a digitized representation of the handwritten 
input divided according to the discrete segments S x and 
S y . Once the recognition of S x and S y is displayed 170, 

25 the user, or input device, may repeat a selection input 
split instruction and may select to split the discrete seg- 
ments S x andS y , if possible. In accordance with the pre- 
ferred method of the present invention a user may con- 
tinue with one or more selection input instructions until 

30 the user no longer selects a split instruction or there are 
no discrete continuos segments remaining to split. 
[0017] Referring now to FIGs 2 through 7, a graphical 
representation of a preferred embodiment of the present 
invention is illustrated. FIGs 2 through 4, illustrate an 

35 input device 1 1 0 upon which is received the handwritten 
input consisting of continuous segments S 2 , and S 3 . 
The continuous segments are recognized as a single 
continuous segment Sa and are displayed to the user 
or input device as such. In accordance with a preferred 

40 embodiment of the present invention as selection input 
is requested by the user, or input device. The selection 
input 125 is a split instruction, as illustrated in FIG. 3. 
FIG 4 illustrates the recognition and display of the dis- 
crete continuous segments S x and S y after the continu- 
es ous segments , S 2 , and S 3 were parsed into strokes 

and processed according to a preferred method sub- 
stantially similar to the method illustrated in FIG 1 . The 
advantage of the present invention allows a user, or in- 
put device to join split an infinite amount of continuous 

so segments S n , S n+A1 , S n+/ . 2 , etc., either repeatedly or si- 
multaneously, to form separate individual discrete seg- 
ments S x , S y , etc. This provides a user, or input device, 
a variety of editing capabilities. 
[0018] FIGs. 5 through 7, illustrate a preferred method 

55 substantially similar to the method illustrated in FIG 1 , 
as steps 125 through 188. In this preferred embodiment, 
the continuous segments S 1: S 2 , and S 3 are parsed into 
strokes T 0 through T end . A map of the smallest gaps be- 
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tween each stroke and all other strokes of the continu- 
ous segments S-), S 2 , and S 3 , is generated. In this map 
the largest value, Z, 125, is recorded for each stroke, 
where Z is equal to the largest value found for each 
stroke, from the list of values of the smallest gaps found 
between that stroke and all other strokes. If map gener- 
ated indicates Z is less than or equal to zero the contin- 
uous segment is found to be a single discrete handwrit- 
ten input. This is illustrated in FIG 5, the continuous seg- 
ment is deemed to be a single discrete handwritten 
input, in this example a single word that should not be 
split. 

[0019] As illustrated in FIG 6 - 7, if however the value 
of Z is greater than zero for any of the strokes T, the one 
or more continuous segments S 2 , and S 3 , are split 
at the stroke number T x having the largest value Z 150; 
where T x is defined as the split stroke number. If a split 
stroke number is found, at least one discrete segment 
S x is generated 1 55, where S x includes the discrete con- 
tin uous segments from stroke T = Zero to the split stroke 
number T x . At least a second discrete segment S y is 
generated 160, where S y includes the discrete continu- 
ous segments from split stroke number plus one, T x+1 , 
to the last stroke T end of the continuous segments 
S 2 , and S 3 . In accordance with the preferred embodi- 
ment of the present invention, the discrete continuous 
segments S x and S y are each analyzed and recognized 
as separate, distinct handwritten inputs by the handwrit- 
ing recognition method executing on the input device 
165. The recognition for both S x and S y is displayed to 
the user or input device 170. 



1 . A method for facilitating recognition of handwritten 
input, comprising the steps of: 
receiving: 

electronic data comprising a series of data 
points that correspond to an original handwritten in- 
put of continuous segments and recognized words 
that correspond to said original handwritten input of 
continuous segments evaluated together: 

displaying the recognized words; 
receiving user selection input comprising a split 
instruction; and upon receiving the split instruc- 
tion: 

automatically determining a split position 
generating discrete continuous segments S x 
and S y on either side of said split position; 
receiving recognized words that correspond to 
each of discrete continuous segments S x and 



displaying said recognized words that corre- 
spond to discrete continuous segments Sx and 
Sy. 



2. A method for facilitating recognition of handwritten 
input according to claim 1 , the method further com- 
prising the steps of: 

parsing the series of data points into a series 
of strokes generating a map of the largest value 
for each of the values of the smallest gaps of 
each stroke as compared to every other stroke; 
wherein 

upon generating a value greater than zero 
splitting said one or more continuous segments 
at a split stroke number T x , said split stroke 
number having the largest value of said maps 
among the total number of strokes; 
generating at least one discrete continuous 
segment S x incorporating the strokes from a 
first stroke T 0 to the split stroke number T x ; and 
generating at least one discrete continuous 
segment S y incorporating the strokes from a 
first stroke equal to split stroke number plus 
one, T x + 1 , to a final stroke equal to T end . 



1. Ein Verfahren zur Erleichterung der Erkennung 
handschriftlicher Eingaben, das die folgenden 
Schritte aufweist: 

Empfang elektronischer Daten, die eine Folge 
von Datenpunkten aufweisen, die einer ur- 
spriinglich handschriftlichen Eingabe aus kon- 
tinuierlichen Segmenten entsprechen, und er- 
kannte Worter, die der ursprijnglich hand- 
schriftlichen Eingabe aus kontinuierlichen Seg- 
menten entsprechen, die zusammen evaluiert 
wurden; 

Anzeige der erkannten Worter; 

Empfang einer Anwenderauswahleingabe, die 
eine Aufteilungsanweisung aufweist; und 

bei Empfang der Aufteilungsanweisung auto- 
matische Bestimmung einer Aufteilungspositi- 
on und Erzeugung einzelner kontinuierlicher 
Segmente S x und S y auf beiden Seiten der Auf- 
teilungsposition; 

Empfang erkannter Worter, die jedem der ein- 
zelnen kontinuierlichen Segmente S x und S y , 
die getrennt evaluiert wurden; und 

Anzeige der erkannten Worter, die jedem der 
einzelnen kontinuierlichen Segmente S x und S y 
entsprechen. 

2. Ein Verfahren zur Erleichterung der Erkennung 
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10 



handschriftlicher Eingaben nach Anspruch 1 , und 
das Verfahren weiter die folgenden Schritte auf- 
weist: 

Zergliedern der Folge von Datenpunkten in ei- s 
ne Folge von Schreibziigen und Erzeugung ei- 
ner Karte des groBten Wertes fur jeden der 
Werte der kleinsten Lukken eines jeden 
Schreibzuges im Vergleich mit jedem anderen 
Schreibzug; wobei 10 

bei Erzeugung eines Wertes groBer Null dieser 
eine oder mehrere kontinuierliche Segmente 
bei einer Aufteilungsschreibzugnummer T x auf- 
geteiltwerden, und die Aufteilungsschreibzug- is 
nummer den groBten Wert der Karten unter der 
gesamten Anzahl von Schreibziigen hat; 

Erzeugung von wenigstens einem einzelnen 
kontinuierlichen Segment S x , das die Schreib- 20 
ziige von einem ersten Schreibzug T 0 bis zur 
Aufteilungsschreibzugnummer T x verkorpert; 
und 

Erzeugung von wenigstens einem einzelnen 25 
kontinuierlichen Segment S y , das die Schreib- 
ztige von einem ersten Schreibzug, welcher 
der Aufteilungsschreibzugnummer plus Eins 
entspricht, T x+1 , bis zu einem letzten Schreib- 
zug, der T ende entspricht, verkorpert. 30 



dent a chacun des segments continus dis- 
crete S x et S y evalues separement ; et 
afficher lesdits mots reconnus qui corres- 
pondent aux segments continus discrets 
S x et S y . 

Precede permettant de faciliter la reconnaissance 
de I'ecriture manuscrite selon la revendication 1 , le 
precede comprenant en outre les etapes consistant 



analyser du point de vue syntaxique la serie de 
points de donnees dans une serie de traits 
generer une carte de la plus grande valeur pour 
chacune des valeurs des plus petits interstices 
de chaque trait par rapport a tous les autres 
traits ; dans lequel 

lors de la generation d'une valeur superieure a 
zero 

fractionner ledit ou lesdits segments continus 
en un nombre de traits fractionnes T x , ledit 
nombre de traits fractionnes ayant la plus gran- 
de valeur desdites cartes parmi le nombre total 
de traits ; 

generer au moins un segment continu discret 
S x incorporant les traits d'un premier trait T 0 au 
nombre de traits fractionnes T x ; et 
generer au moins un segment continu discret 
S y incorporant les traits d'un premier trait egal 
au nombre de traits fractionnes T x+1 a un trait 
final egal a T fin . 



1. Precede permettant de faciliter la reconnaissance 35 
de I'ecriture manuscrite, comprenant les etapes 
consistant a : 
recevoir : 

des donnees electroniques comprenant une 40 
serie de points de donnees qui correspondent 
a une entree manuscrite originale de segments 
continus et de mots reconnus qui correspon- 
dent a ladite entree manuscrite originale de 
segments continus evalues ensemble ; « 
afficher les mots reconnus ; 
recevoir I'entree de selection de I'utilisateur 
comprenant une instruction de fraction- 
nement ; et 

lors de la reception de I'instruction de so 
fractionnement : 

determiner automatiquement une position 
de fractionnement ; 

generer des segments continus discrets S x 55 
et S y de chaque cote de ladite position de 
fractionnement ; 

recevoir des mots reconnus qui correspon- 
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RECEIVE HANDWRITTEN INPUT Cf ONE 
OR MORE CONTINUOUS SEGMENTS, 



RECOGNIZE HANDWRITTEN INPUT AS ONE OR 
MORE CONTINUOUS SEGMENTS 

S N ,S Nf1' S M t 2-" IS 1' S 2' S 3 1 



□SPUYRECONIZEOFS^.S^... 
AS ONE OR MORE CONTINUOUS 



Input split continuous^ 

B M ,S KI . H ,S M JNTOONE 



AS ONE OR MORE CONTINUOUS SEGMENTS OF p 
HANDWRITTEN INPUT 



GENERATE A MAP OF THE SMALLEST GAPS 
BETWEEN EACH STROKE AND ALL OTHER STROKES 
OF THE ONE OR MORE CONTINUOUS SEGMENTS 



FIND THE LARGEST VALUE Z OF THE MAP 
OF SMALLEST GAPS BETWEEN EACH 
STROKE AND ALL OTHER STROKES ... 




SPLIT ONE OR MORE CONTINUOUS SEGMENTS 

AT THE STROKE NUMBER HAVING THE 
LARGEST VALUE Z [THE SPUT STROKE NUMBER] 



figja 
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GENERATE DISCRETE SEGMENT S v INCLUDE 



DISCRETE CONTINUOUS SEGMENT FROM STROKE = C 
TO SPLIT STROKE NUMBER 



160~ 



GENERATE DISCRETE SEGMENTS y INCLUDE 

CONTINUOUS SEGMENT FROM STROKE = [SPUT 
STROKE NUMBER+1] TO THE LAST 
STROKE OF CONTINUOUS SEGMENT 



T 



RECOGNIZE S ^ AND S y t 
DISCRETES 


\S DISTINCT INDIVIDUAL 
EGMENTS 






DISPLAY RECOGNITION OF S x AND S y | 
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SELECTION 
INPUT SPUT CONTINUOUS 
SEGMENTS K „S 



YES 

FIG.1B 
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DISPLAY RECOGNITION OF S N = S y/W S = S y 
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